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I. INTRODUCTION 



The reliability of any system composed of N components is defined: 

R(T^) = Probability (T > T^) 

where T denotes the time of first system failure. A significant practical 
problem is that it is often reasonably easy, and comparatively inexpensive, 
to determine information about the reliability of separate component types, 
denoted ^ N , but fairly difficult and extremely expen- 

sive to determine total system* reliability directly, since failure testing 
often destroys the system. 

In the most common analytically treated case, statistically independent 
components in series, where each of the component types has an assumed ex- 
ponential distribution of failures, i.e., 

-X.T 

Ri<To) = e ^ i = 1,2,..., N , (1) 

the system reliability is given by 

-XT N 

R(T ) = e X = ^ X . (2) 

i=l 



(It should be noted in complex systems that these are valid only when all 
components operate the same fraction of mission time.) 

A statistically exact procedure for obtaining upper bounds on X from 
data on component failure, here called the LR procedure, has been developed 
by Lieberman and Ross (1971). In this procedure items of each type 

component, with individual failure times ^ ^ ” I 52 ,..., N and 

j = 1,2,..., are to be observed. A time U is defined: 



U = min. 

1 



k. 

L T_. 



j=i 



ij 






(3) 



1 



i.e«, U is the cumulative time at which one first exhausts all the 
components of one type. The number of each type of component which has 
been used up to this time if given by: 



. largest {j < | 



(4) 



Then , if 



K 



N 

I ”i 



(5) 



i=l 



(i.e., total components used), Lieberman and Ross showed that 2XU follows 



the X 2 j^ distribution; hence upper bounds for X can be estimated. 



A second procedure for estimating upper bounds on X is the approximal 
optimum procedure for Type II censoring (fixed number of failures) developec 
by Mann and Grubbs (1974) , which approximates the optimum exact 
bound of El Mawaziny (1965) . In this Mann and Grubbs (hereafter referred 
to as MG) procedure, it is assumed that testing of all components of type 
i commences simultaneously, and continues until r^ of the original 

have failed. (It is permissible that Then, if the individual 



failure times are ordered so that T , , < T..,,,x, the total test time for | 

ij ” i(j+l) 



.th 



the i component type is defined. 



. = y T. . + (k. - r.)T 
1 13 1 1 



ir. 



( 6 ) 



j=l - 1 

The approximate optimum upper bound for X, at confidence level a, 

is given by: 



,MG ) V 

X, = m { 1 TT + 

' 9m^ 3m 



(7) 



2 



where 

and 

and 



is the 100a 



th 



percentile of the standard normal distribution, 



N 

m = I {(r - 1)/Z } + 1/U , 

i=l 

N , , 

V I {(r. - 1)/Z. } + 1/lT , 
i=l 



( 8 ) 

(9) 



with U defined by (3) above. 

A major drawback of the LR technique is that in using K as 
defined by (4)- (5), one "discards" the data on the 



N 

NL = ^ {k - n } (10) 

i=l 

components which do not fail by the time U. While necessary for the 

statistical validity of the procedure, this means that, effectively, 

the LR procedure assumes components of each type are tested sequentially, 

s t 

i.e. testing of the (j+1) component of type i commences with failure 
of j component of that type. This is seldom the testing scheme in 
practice. 

An immediate consequence of this and (4) is that the LR procedure 
is Data-Order Dependent , that is, permuting the second subscript on 
(i.e., in essence permuting the order in which the failures are 
observed) can alter n^, and hence the LR estimated bound. This can 
lead to significant practical problems, for often failure data on individual 
components is gathered in simultaneous (parallel) testing of all 
components, rather than in sequential testing. Thus there is no preferred 
ordering, and, before using the LR procedure, the analyst is faced with 
the formidable task of deciding on the *'best” way to assign the T.., 



3 



It is obvious that the MG procedure does not suffer these drawbacks. 

(Note that the data order dependence of the LR method is a mixed blessing, 
since it may allow the analyst to construct several alternative bounds to 
X, all of equal statistical validity, by simply permuting the data.) 

The purpose of this paper is to compare the LR and MG procedures 
in terms of the expected bounds, and effects of "discarded" data, using 
both analytic and simulation techniques. 
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II. COMPARISON OF THE TWO METHODS -ANALYTIC RESULTS 



Immediate comparison of the LR and MG procedures is complicated by the 

fact the test schemes are conceptually different. As noted above, the deri- 

s t 

vation of the LR procedure presupposes that testing of the (j + 1) com- 
ponent of any type begins with the failure of the component of that 

type, and that all component testing ceases at U , with the exhaustion of 
the supply of one type of component. Thus, conceptually, the LR ends with 
(N - 1) components (one of each of the other types) still undergoing testing, 
and no failed plus others, (NL + 1 - N) to be exact, not even tested. By 
contrast, the MG procedure implicitly assumes, for each i , testing of all 
components of type . i starts simultaneously, and terminates with a fail- 
ure of some one of them. Thus before data gathered according to the MG 
procedure can be utilized to produce a LR bound, the analyst must first 
choose one (or more) a priori orderings of the data. 

However, for exponential failure data, we appeal to the constant failure 
rate property to convert data gathered in a LR test format into MG format. 
Precisely, constant failure rate implies that irrespective of whether testing 
of the i^^ component type terminates at a failure, the sufficient statistics 
are total test time and number of failures observed in that time. Thus, if 
all testing is halted at U according to the LR model, then, we shall apply 
the MG procedure (providing each component type experienced at least one 
failure) with: 



Z. = U 
1 

r. = n. > 1 



i = 1,2 



, N 



( 11 ) 



1 1 / 

(This does mildly violate the MG assumptions, in that the n^ is random, not 

set a priori. However, any effect sould be noticeable only in extremely small 

MG 

sample sizes, and will not affect our basic conclusion unless < X} is 

significantly increased as a result.) 
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From this, it immediately follows that 



N 



m = I {(n - 1)/U} + 1/U 

i=l 



( 12 ) 



N ) 

I (n - 1) + 1 J 

;i=i i ) 

where K is given by (5), and, similarly, 



/U = (K + 1 - N)/U, 



V - (K + 1 - N)/U . 



(13) 



But, with these values, (7) reduces to 
^MG ^ (K + 1 - N) j ^ _ 



U 



1 I 

j 9(K + 1 - N) 3^(K + 1 - N) I 



. (14) 



Observe (14) is a strictly increasing function of (K+l-N) for U 
fixed and < 5.65 (which includes all cases of practical interest). 



Thus , since N ^ 2 , 



,MG ^ (K-1) , . 



1 . 1 
3 vie - 1 I 



(15) 



Since the bound produced by the LR method from this data is 



,LR 



2U 



(16) 



we see that, when the MG procedure is applied to the same data actually 
used in the LR procedure , the computed bounds satisfy. 






9(K - 1) 3/k - 1 



(17) 
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In Table 1, we show values of the right hand side of (17) for representative 
K and a. It is obvious that, for all practical values of K and a. 



( 





) 



< 



1 . 



Thus, when both the Liebennan-Ross and Mann-Grubbs procedures are 
used to compute an upper bound to X, if the MG procedure uses only 
the same failure times as are actually used by the LR procedure, the upper 
bound produced by the Mann Grubbs procedure will be smaller than that 
produced by Lieberman-Ross. 

An immediate extension of the above result is that the LR procedure 
may produce lower bounds than the MG procedure only in those cases where 
the LR procedure does not utilize all the data available. Specifically, 
consider the situation where the LR procedure utilizes failure times on 
all but one component of each type except the type which exhausts first, 
which we assume is type i = 1 . Testing is assumed to have continued 
on these (N - 1) components until they also failed. Thus: 



"l 
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n^ ^i ” ^ 2, ..., N, 



(18) 



and 



= U, 



> U, i = 2, ..., N. 



(19) 



Let e > 0 and arbitrarily small, and assume that each component of type 
i _> 2, on test at t = U, fails at t = U + e. Thus, MG can be applied 
with: 



7 



r. = k. 



i = 1, 2, .... N,' 



Z = U 



( 20 ) 



Z^(e) 



U + e, i = 2, . , N. 



Sine 



e, for e arbitrarily small, Z^(e) < Z^, i = 2, N, 



we are thus understating the reliability of components of type i, i ^ 2. 

MG 

Hence we expecte the bound computed using Z^(e), denoted (e) to 
be conservative, i.e. 

,MG ^ -MG. - 
Xb < X^ (e). 



and, more specifically 



,MG ^ -MG. - 



( 21 ) 



It is easily shown 



0^ iMG 



(c) -is li + 



with. 



( 22 ) 



N N N 

K = I (r^ - 1) + 1 = I (k^ - 1) + 1 = J (k^ - 1) , 



i=l ^ i=l " " i=2 

i.e. K is degree of freedom from the LR procedure on this data. Thus, 
for this data. 



(xf/x“)< 






I ^ 3*^ I 



(23) 



Computations show that, for .75 ^ ct ^ .995, and 2 ^ k £ 120, the 
righthand side of (23) does not differ from unity by more than 0.4%, except 
k = 2 and a = .995, when it differs from unity by 0.8%. (Some of 
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this may be due to minor inaccuracies in the computer routines used.) 

Thus, we conclude that the Lieberman-Ross method can not produce a signifi-- 
cantly lower, bound on X than the Mann Grubbs method, unless individual 

component failure times ara collected, separately, and combined into pseudo-system 
failures in such a way that the LR procedure assumes some of the components 
were never tested, i.e. the LR method must deliberately discard some of 
the data in order to compute a better bound than MG. 

Since both the Lieberman-Ross and Mann-Grubbs procedures estimate 
upper bounds to X, it follows that, unless the confidence level (a) is 
sufficiently low that computing a false bound is significantly probable, 
the method producing the lower estimated bound will be superior. Thus, 
we look for the LR method to be superior to MG when the ordering of 
the data chosen for the LR method is such that some, probably a 
significant fraction, of the available data on failure times is not 
utilized, i.e. when NL, as defined by (10) satisfies NL > (N-1) . 
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Ill COMPARISON OF THE TWO METHODS - SIMULATION RESULTS 

Having determined that the cases where the Lieberman-Ross method 

should be superior to the Mann-Grubbs procedure should arise among those 

where the data ordering used for the LR procedure "discards" some of 

the available failure data, we preceded to investigate this relationship 

in more detail. We felt that the most useful measure would be the 

probability that the LR method would produce a superior bound, as a 

fxmction of the amoxint of data unused. Since analytic techniques were 

not able to shed much additional light in this area, we turned to simulation, 

The simulation described in [4], which allows generation of component 

failure data for up to twenty individual components of up to six different 

types, was modified for this purpose. We restricted ourselves to the I 

two component type (N=2) case for simplicity. Four individual runs, 

each run consisting of 500 iterations, were conducted for each of the 

combinations of confidence levels, components and failure rates shown in 

Table 2. In each iteration, independent exponential failure times were 

generated for all the components specified of each type, and the MG 

MG 

procedure applied to these data, to produce a bound, , at the 

specified confidence limits. Then the data were ordered into a single 

LR pseudo- failure sequence. The manner in which individual component 

failures were generated seirved to insure this was a suitably random 

ordering of the failure times. The LR procedure was applied to this 

LR 

sequence, to produce the bound Xj^ . The two bounds were compared, 
and the LR procedure deemed to produce the superior bound if either of 
the following held: 
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( 1 ) 



9 



X < < xf 

— b b 



(2) x“^ < X £ X^^ , 



(3) X^^ < X^^ < X . 



In each iteration, this result was comparetf with the percentage of 
available (i.e. generated) data not used by the LR procedure: 



N 

PL = { I (k 
i=l 



i 



- n.)}/{ 



N 



I k. } , 
iii ^ 
i+f 



(24) 



where f is the index of the component type which was first exhausted 

i.e. • 

At the completion of each 500 iteration run, the percentage of 
iterations where the data ordering had produced a superior bound by the 
LR method was computed, plotted, and tabulated as a function of the 
percentage of lost data, PL. The results, for each run, were given in 
a function: 



TOTR(J) = 



The percentage of data orderings 
with .05(J-1) < PL £ .05 J where 
the LR method produced a superior 
bound . 



(25) 



(Note TOTR(J) was set to -1.0 if no data orderings with 
.05(J-1) < PL £ .05J were observed on that run.) The plotted distribution 
of values of TOTR(J) for runs at a = .95 and a = .80 are shown at 
Figures 1 and 2 respectively. Since TOTR(J) can be regarded as an 
estimate to the probability that the LR method produces a superior 
bound, given that between 5(J-1)% and 5J% of the data was ignored. 
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it is obvious that only if the analyst is willing to order the data 

in such a way that 80% or more of the available data is not used, will 

the probability that the LR bound is superior likely exceed one half. 

(The reason for the superiority of the LR procedure in some cases when 

little data was discarded is that, in these cases, the MG procedure 

produced a false bound, i.e. < X .) 

From Figures 1 and 2 it appears the LR procedure performs slightly 

better vis-a-vis the MG procedure at a = .80 than at a = .95. Essential]^ 

MG 

this is the result of the probability that X^ < X increasing as a 
decreases. 

Based on our observation that an analyst would have to order LR 
data in such a way that the amount of discarded data exceeded 80% of 
that available before the LR could be reasonably expected to out perform 
MG, the best strategy to do this would be to pick the in decreasing 

order, i.e. 



^i(j+l) ^ ^ij ’ 



i=l, 2, 

This is precisely equivalent to discarding, as much as possible, the 
least reliable samples of each component type. This is not, philosophicalli 
a particularly attractive strategy. 
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IV. SUMMARY AND CONCLUSIONS 



In this paper we have examined two different methods for computing 
bounds on series system reliability, the statistically exact procedure of 
Lieberman and Ross, and the approximately optimum procedure of Mann and 
Grubbs. Our analysis showed that, when used with the same data, the MG 
procedure would always produce a lower upper bound on X than the LR procedure. 
Thus, the MG procedure seems preferable, especially at high confidence levels, 

since the probability of computing a false bound is correspondingly small 
there. Further analysis and simulation results showed that the LR 
procedure could be reasonably expected to produce a superior bound to MG 
only where the failure times were ordered such that the LR procedure 
essentially "discarded** most of the data, and this is usually equivalent 
to retaining samples of only the most reliable components. 

Thus, it appears, at present, that using the LR procedure would be 
preferable to using the MG procedure only in very special circumstances. 

The only qualification we should note is that we have not fully utilized 
the LR procedure's ability to usually produce multiple bounds from the 
same data. This may be exploited in some way to make the LR procedure 
more competitive, however we are not certain of that at this time. 
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TABLE 2 



A. Sample Sizes Used in Simulation 



k 



1 



k 



2 



2 

2 

2 

2 

3 

3 

3 

5 

5 

10 



2 

3 

4 

5 

3 

4 

5 
5 

10 

10 



B. Ratio of Failure Rates Used in Simulation 

(Xj^/X2> = 10.00, 5.00, 2.50, 1.25, 1.00, 0.500, 0.250, 0.125, 



For the two component case , 



(X^/X 2 ) is only rate needed. 



0.100 
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